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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
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Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )KI Responsive to communication(s) filed on 20 August 201 0 . 
2a )^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-15 and 17-19 is/are pending in the application. 

4a) Of the above claim(s) 2-8,10-13 and 18 is/are withdrawn from consideration. 

5) ^ Claim(s) 19 is/are allowed. 

6) |EI Claim(s) 1,9,14 and 17 is/are rejected. 

7) ^ Claim(s) 15 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 01 August 2006 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
IDS 

[01] The IDS filed 8/20/10 is blank. 

Specification 

[02] The objections to the specification set forth in the Office action of 5/20/10 are withdrawn in 

view of the amendments of 8/20/10. 

Claim Rejections - 35 USC § 112, Paragraph One 
[03] The text of those sections of Title 35, U.S. Code not included in this action can be found in 

a prior Office action. 

[04] The rejection of claim(s) 1 6 set forth in the Office action of 5/20/10 is withdrawn in view of 

the arguments of 8/20/10. 

Claim Rejections - 35 U.S.C. 112, Paragraph Two 
[05] The text of those sections of Title 35, U.S. Code not included in this action can be found in 

a prior Office action. 

[06] The rejection of claim(s) 9, 14-16 set forth in the Office action of 5/20/10 are withdrawn in 

view of the amendments of 8/20/10. 

Claim Rejections - 35 USC § 103 
[07] The text of those sections of Title 35, U.S. Code not included in this action can be found in 

a prior Office action. 

[08] The rejection of claim(s) 1 ,9,14 as being unpatentable over Gombrich (6,081 ,740) in view of 
Heung-Yeung Shum as set forth in the Office action of 5/20/10 are withdrawn in view of the 
amendments of 8/20/10. 
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[09] Claim(s) 1 ,9,14,17 is/are rejected under 35 U.S.C. 103(a) as being unpatentable over Iddan 
(2004/0249247) in view of Heung-Yeung Shum and R. Szeliski, "Construction and 
Refinement of Panoramic Mosaics with Global and Local Alignment," Proc. Int'l Conf. 
Computer Vision, pp. 953-958, 1998. 

[1 0] With regard to claim 1 : Iddan discloses an endoscope system for taking images of an inside 
of an object, comprising: 

[1 0a] an omnidirectional camera ("image sensor 208" [001 6]) operable to take a plurality 
of images of the inside of the object in a living body, which is capable of motion; 
and 

[1 0b] an image generation unit ("a processor" operable to generate a panoramic image 
of the inside of the object by performing a video mosaicking process [] on the 
plurality of images obtained by said omnidirectional camera, said processes being 
intended for pasting the images, estimating camera motion, correcting previously 
definable motion in the living body and correcting previously indefinable internal 
deformation in the living body; 

[10c] wherein said image generation unit is operable to generate the panoramic image 

such that the panoramic image has a fixed visual angle with respect to each of 

directions perpendicular to a traveling direction of said omnidirectional camera, by 

performing the video mosaicking process on the plurality of images obtained by 

said omnidirectional camera . 

[001 8] In one embodiment, as tube 200 of endoscope 201 traverses a 
body lumen 205, imaging unit 206 may capture images of a slice 207 
of body lumen 205. For example, a ring shaped slice of a portion of 
tissue located behind the distal tip of the endoscope 201 may be 
imaged. The ring shaped slice may encompass a 360 degree view 
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surrounding the circumference of the endoscope. Illumination unit 
21 2 may illuminate the slice 207 of body lumen 205. The light from 
the illuminated slice 207 may be reflected by reflective surface 214, 
focused by lens 210, and received by imager 208 which may capture 
an image of the slice 207 of body lumen 205 from an area 
surrounding tube 200 of endoscope 201 . In an embodiment, since 
tube 200 may include transparent segments 216 in the shape of a 
ring of tube 200, the captured image may include a reflected image of 
a ring-shaped slice 207 of body lumen 205. It is noted that lens 2 1 0 
may be configured, placed and/or aligned to filter and/or focus fight 
from body lumen 205, such that only light from a desired portion of 
body lumen 205, for example, a ring-shaped slice 207, falls on image 
sensor 208. The image captured by image sensor 208 may include a 
substantially complete 360 degrees image of a slice of body lumen 
205. Alternatively slices may include other configurations, for 
example, a 270 degree image, a 21 0 degree image, a 1 80 degree 
image, or any other number of degrees between 0 and 360. 

[001 9] In some embodiments, the panoramic image of a slice 207 of 
body lumen 205 may be ring-shaped. Such an image may be 
recorded as or converted into a rectangular image or into other 
shapes. In one embodiment, the conversion may be performed, for 
example, by a processor that may be included in endoscope 201 or 
that may be housed in a separate unit. In some embodiments, 
processor 234 or another processor may convert images into a 
rectangular image. The conversion may be performed, for example, 
using methods as known in the art to "flatten" a ring-shaped image 
into a rectangular image. The conversion may be applied to one 
image, or to a group or a batch of sequential or non-sequential 
images. In some embodiments, one or more of a series of panoramic 
images of slices 207 of body lumen 205 may be connected to form a 
moving image of body lumen 205 or a sequential chain of images of 
slices 207 of body lumen 205. In some embodiments a video image 
may be captured. In some embodiments, depending on the 
configuration of one or more reflecting surfaces, the image as 
recorded on an imager may be substantially rectangular. 

[0020] Additionally or alternatively, images of slices 207 of body 
lumen 205, may be placed, aligned and/or combined together, for 
example, side by side, to create a combined image or several 
combined images from a plurality of images of slices 207. The 
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combination of images of slices 207 may be performed, for example, 
by a processor as is discussed herein. 

[1 Od] Iddan does not disclose a motion correction process or an image modification 
process through energy minimization on the plurality of images obtained by said 
omnidirectional camera. 

[10e] Heung-Yeung (hereafter "HY") discloses a motion correction process ("motion 

estimate" p957), and an image modification process through energy minimization 
on the plurality of images obtained by said camera ("minimize the difference 
between screen coordinates of all overlapping pairs of images" p955), said 
processes being intended for estimating camera motion ("camera translation" 
p957), correcting previously definable motion in the living body and correcting 
previously indefinable internal deformation in the living body ("moving objects") 
p957. 

[1 Of] At the time of the invention, it would have been obvious to a person of ordinary 
skill in the art to provide motion correction and image modification through energy 
minimization on the images captured by Iddan's device. A skilled artisan would be 
motivated to do so in order to generate a panoramic image that is free from 
misregistrations caused by camera translation or moving objects, as taught by HY. 
[11] With regard to claim 9: the image generation unit disclosed by Iddan in view of HY 

comprises: 

[11a] a feature region cutout unit operable to cut out a plurality of feature regions ("use 
the patch centers as prospective "feature" points" p955 of HY) having a 
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predetermined size ("e.g., 16x16 pixels" p955) from each of the plurality of 
images obtained by said omnidirectional camera: and 
[11b] a panoramic image generation unit operable to define an energy based on the 

plurality of feature regions included in each of the plurality of images, associate the 
plurality of feature regions between the plurality of images such that the energy is 
minimized, and generate a panoramic image of the inside of the object based on 
the association result ("minimize the difference between the ray directions of 
corresponding points using a rotational panoramic representation with unknown 
focal length" p955). 

[12] With regard to claim 14: the predetermined energy is determined based on a degree of 
deviation between a plurality of control points selected from a first image taken by said 
camera and a plurality of control points, in a second image taken by said camera, which 
respectively correspond to the plurality of control points selected from the first image 
("corresponding points between pairs of images are automatically obtained using patch- 
based alignment" p955). 

[1 3] With regard to claim 1 7: said omnidirectional camera is mounted on a tip of a probe that is 
to be inserted into a digestive organ. 

Allowable Subject Matter 

[14] Claim(s) 19 is/are allowed. 

[1 5] Claim(s) 1 5 is/are would be allowable if rewritten to overcome the rejection(s) under 35 
U.S.C. 112, 2nd paragraph, set forth in this Office action and to include all of the limitations 
of the base claim and any intervening claims. 

Response to Arguments 
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[1 6] Applicant's arguments filed 8/20/1 0 have been fully considered but they are not persuasive. 
[1 7] Applicant's arguments have been considered but are moot in view of the new ground(s) of 
rejection. 

Conclusion 

[1 8] Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is 
reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

[19] A shortened statutory period for reply to this final action is set to expire THREE MONTHS 
from the mailing date of this action. In the event a first reply is filed within TWO MONTHS 
of the mailing date of this final action and the advisory action is not mailed until after the 
end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .1 36(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of 
this final action. 

[20] Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to PHILIP R. SMITH whose telephone number is (571)272-6087 and 
whose email address is philip.smith@uspto.gov. The examiner can normally be reached 
between 9:00am and 5:00pm. 

[21] If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Linda Dvorak can be reached on (571) 272 4764. 

[22] Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
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applications may be obtained from either Private PAIR or Public PAIR. Status information 
for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on 
access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866- 
217-9197 (toll-free). 
/Philip R Smith/ 

Primary Examiner, Art Unit 3779 
/Thomas J Sweet/ 

Supervisory Patent Examiner, Art Unit 3779 



